1.3.1 How not to visualize data

Below are some examples of visualizations that could be improved. How would you make them better?

1.3.2 How to make it better

In this class, we you will learn how to use ggplot2 to make nice figures. The ggplot2 library provides a unified framework for making plots – it defines a grammar of graphics according to which we construct figures step by step.

Instead of learning rigid rules for what makes for a good figure, you will learn how to make figures yourself, play around with things, and get a feeling for what works best.

1.6.1 Setting up a plot

Let’s first get some data.

df.diamonds = diamonds

The diamonds dataset comes with the ggplot2 package. We can get a description of the dataset by running the following command:

?diamonds

Above, we assigned the diamonds dataset to the variable df.diamonds so that we can see it in the data explorer.

Let’s take a look at the full dataset by clicking on it in the explorer.

Tip: You can view a data frame by highlighting the text in the editor (or simply moving the mouse above the text), and then press F2.

The df.diamonds data frame contains information about almost 60,000 diamonds, including their price, carat value, size, etc. Let’s use visualization to get a better sense for this dataset. We start by setting up the plot. To do so, we pass a data frame to the function ggplot() in the following way.

ggplot(data = df.diamonds)

This, by itself, won’t do anything yet. We also need to specify what to plot.

Let’s take a look at how much diamonds of different color cost. The help file says that diamonds labeled D have the best color, and diamonds labeled J the worst color. Let’s make a bar plot that shows the average price of diamonds for different colors.

We do so via specifying a mapping from the data to the plot aesthetics with the function aes(). We need to tell aes() what we would like to display on the x-axis, and the y-axis of the plot.

ggplot(data = df.diamonds,
       mapping = aes(x = color, y = price))

Here, we specified that we want to plot color on the x-axis, and price on the y-axis. As you can see, ggplot2 has already figured out how to label the axes. However, we still need to specify how to plot it. Let’s make a bar graph:

ggplot(data = df.diamonds,
       mapping = aes(x = color, y = price)) +
  stat_summary(fun.y = "mean", geom = "bar")

Neat! Three lines of code produce an almost-publication-ready plot (to be published in the Proceedings of Unnecessary Diamonds)! Note how we used a + at the end of the first line of code to specify that there will be more. This is a very powerful idea underlying ggplot2. We can start simple and keep adding things to the plot step by step.

We used the stat_summary() function to define what we want to plot (the “mean”), and how (as a “bar” chart). Let’s take a closer look at that function.

help(stat_summary)

Not the the easiest help file … We supplied two arguments to the function, fun.y = and geom =.

1. The fun.y argument specifies what function we’d like to apply to the data for each value of x. Here, we said that we would like to take the mean and we specified that as a string.
2. The geom (= geometric object) argument specifies how we would like to plot the result, namely as a “bar” plot.

Instead of showing the “mean”, we could also show the “median” instead.

ggplot(data = df.diamonds, mapping = aes(x = color, y = price)) +
  stat_summary(fun.y = "median", geom = "bar")

And instead of making a bar plot, we could plot some points.

ggplot(df.diamonds, aes(x = color, y = price)) +
  stat_summary(fun.y = "mean", geom = "point")

Tip: Take a look here to see what other geoms ggplot2 supports.

Somewhat surprisingly, diamonds with the best color (D) are not the most expensive ones. What’s going on here? We’ll need to do some more exploration to figure this out.

Note that in the last plot, I removed the data = and mapping = specifiers. These keywords are optional, and as long as we provide the arguments to the function in the correct order, we are ok. So, the following doesn’t work:

ggplot(aes(x = color, y = price), df.diamonds) +
  stat_summary(fun.y = "mean", geom = "point")

While this works:

ggplot(mapping = aes(x = color, y = price), data = df.diamonds) +
  stat_summary(fun.y = "mean", geom = "point")

In general, it’s good practice to include the specifiers – particularly for functions that are not used all the time. If the same function is used multiple times throughout the script, I would suggest to use the specifiers first, and then it’s ok to drop them later.

1.6.2 Setting the default plot theme

Before moving one, let’s set a different default theme for our plots. Personally, I’m not a big fan of the gray background and the white grid lines. Also, the default size of the text should be bigger. We can change the default theme using the theme_set() function like so:

theme_set(
  theme_classic() + # set the theme
    theme(text = element_text(size = 20)) # set the default text size
)

From now onwards, all our plots will use what’s specified in theme_classic(), and the default text size will be larger, too. For any individual plot, we can still override these settings.

1.6.3 Scatter plot

I don’t know much about diamonds, but I do know that diamonds with a higher carat value tend to be more expensive. color was a discrete variable with seven different values. carat, however, is a continuous variable. We want to see how the price of diamonds differs as a function of the carat value. Since we are interested in the relationship between two continuous variables, plotting a bar graph won’t work. Instead, let’s make a scatter plot. Let’s put the carat value on the x-axis, and the price on the y-axis.

ggplot(data = df.diamonds, mapping = aes(x = carat, y = price)) +
  geom_point()

Cool! That looks sensible. Diamonds with a higher carat value tend to have a higher price. Our dataset has 53940 rows. So the plot actually shows 53940 circles even though we can’t see all of them since they overlap.

Let’s make some progress on trying to figure out why the diamonds with the better color weren’t the most expensive ones on average. We’ll add some color to the scatter plot in Figure 1.15. We color each of the points based on the diamond’s color. To do so, we pass another argument to the aesthetics of the plot via aes().

ggplot(data = df.diamonds,
       mapping = aes(x = carat,
                     y = price,
                     color = color)) +
  geom_point()

Aha! Now we’ve got some color. Notice how in Figure 1.16 ggplot2 added a legend for us, thanks! We’ll see later how to play around with legends. Form just eye-balling the plot, it looks like the diamonds with the best color (D) tended to have a lower carat value, and the ones with the worst color (J), tended to have the highest carat values.

So this is why diamonds with better colors are less expensive – these diamonds have a lower carat value overall.

There are many other things that we can define in aes(). Take a quick look at the vignette:

vignette("ggplot2-specs")

1.6.4 Line plot

What else do we know about the diamonds? We actually know the quality of how they were cut. The cut variable ranges from “Fair” to “Ideal”. First, let’s take a look at the relationship between cut and price. This time, we’ll make a line plot instead of a bar plot (just because we can).

ggplot(data = df.diamonds, mapping = aes(x = cut, y = price)) +
  stat_summary(fun.y = "mean", geom = "line")
#> geom_path: Each group consists of only one observation. Do you need to
#> adjust the group aesthetic?

Oops! All we did is that we replaced x = color with x = cut, and geom = "bar" with geom = "line". However, the plot doesn’t look like expected (i.e. there is no real plot). What happened here? The reason is that the line plot needs to know what points to connect. The error message tells us that each group consists of only one observation. Let’s adjust the group asthetic to fix this.

ggplot(data = df.diamonds, mapping = aes(x = cut, y = price, group = 1)) +
  stat_summary(fun.y = "mean", geom = "line")

By adding the parameter group = 1 to mapping = aes(), we specify that we would like all the levels in x = cut to be treated as coming from the same group. The reason for this is that cut (our x-axis variable) is a factor (and not a numeric variable), so, by default, ggplot2 tries to draw a separate line for each factor level. We’ll learn more about grouping below (and about factors later).

Interestingly, there is no simple relationship between the quality of the cut and the price of the diamond. In fact, “Ideal” diamonds tend to be cheapest.

1.6.5 Adding error bars

We often don’t just want to show the means but also give a sense for how much the data varies. ggplot2 has some convenient ways of specifying error bars. Let’s take a look at how much price varies as a function of clarity (another variable in our diamonds data frame).

ggplot(data = df.diamonds,
       mapping = aes(x = clarity, y = price)) +
  stat_summary(fun.data = "mean_cl_boot",
               geom = "linerange") + # plot bootstrapped error bars first
  stat_summary(fun.y = "mean",
               geom = "point") #add points with means

Here we have it. The average price of our diamonds for different levels of clarity together with bootstrapped 95% confidence intervals. How do we know that we have 95% confidence intervals? That’s what mean_cl_boot() computes as a default. Let’s take a look at that function:

help(mean_cl_boot)

Remember that you can just select the text (or merely put the cursor over the word) and press F1 to see the help. The help file tell us about the function smean.cl.boot in the Hmisc package. The mean_cl_boot() function is a version that works well with ggplot2. We see that this function takes as inputs, the confidence interval conf.int, the number of bootstrap samples B, and some other ones that we don’t care about for now. So let’s make the same plot again with 99.9% confidence intervals, and 2000 bootstrap samples.

ggplot(data = df.diamonds, mapping = aes(x = clarity, y = price)) +
  stat_summary(fun.data = "mean_cl_boot",
               geom = "linerange",
               fun.args = list(conf.int = .999, B = 2000)) + # plot bootstrapped error bars first
  stat_summary(fun.y = "mean",
               geom = "point") #add points with means

Note how the error bars are larger now compared to Figure 1.18 . Note the somewhat peculiar way in which we supplied the parameters to the mean_cl_boot function. The fun.args argument takes in a list of arguments that it then passes on to the function mean_cl_boot.

1.6.6 Grouping data

Grouping in ggplot2 is a very powerful idea. It allows us to plot subsets of the data – again without the need to make separate data frames first.

Let’s make a plot that shows the relationship between price and color separately for the different qualities of cut.

ggplot(data = df.diamonds,
       mapping = aes(x = color,
                     y = price,
                     group = cut)) +
  stat_summary(fun.y = "mean", geom = "line")

Well, we got some separate lines here but we don’t know which line corresponds to which cut. Let’s add some color!

ggplot(data = df.diamonds,
       mapping = aes(x = color,
                     y = price,
                     group = cut,
                     color = cut)) +
  stat_summary(fun.y = "mean",
               geom = "line",
               size = 2)

Nice! In addition to adding color, I’ve made the lines a little thicker here by setting the size argument to 2.

Grouping is very useful for bar plots. Let’s take a look at how the average price of diamonds looks like taking into account both cut and color (I know – exciting times!). Let’s put the color on the x-axis and then group by the cut.

ggplot(data = df.diamonds,
       mapping = aes(x = color,
                     y = price,
                     group = cut,
                     color = cut)) +
  stat_summary(fun.y = "mean", geom = "bar")

That’s a fail! Several things went wrong here. All the bars are gray and only their outline is colored differently. Instead we want the bars to have a different color. For that we need to specify the fill argument rather than the color argument! But things are worse. The bars currently are shown on top of each other. Instead, we’d like to put them next to each other. Here is how we can do that:

ggplot(data = df.diamonds,
       mapping = aes(x = color,
                     y = price,
                     group = cut,
                     fill = cut)) +
  stat_summary(fun.y = "mean",
               geom = "bar",
               position = position_dodge())

Neato! We’ve changed the color argument to fill, and have added the position = position_dodge() argument to the stat_summary() call. This argument makes it such that the bars are nicely dodged next to each other. Let’s add some error bars just for kicks.

ggplot(data = df.diamonds,
       mapping = aes(x = color,
                     y = price,
                     group = cut,
                     fill = cut)) +
  stat_summary(fun.y = "mean",
               geom = "bar",
               position = position_dodge(width = 0.9),
               color = "black") +
  stat_summary(fun.data = "mean_cl_boot",
               geom = "linerange",
               position = position_dodge(width = 0.9))

Voila! Now with error bars. Note that we’ve added the width = 0.9 argument to position_dodge(). Somehow R was complaining when this was not defined for geom “linerange”. I’ve also added some outline to the bars by including the argument color = "black". I think it looks nicer this way.

So, still somewhat surprisingly, diamonds with the worst color (J) are more expensive than dimanods with the best color (D), and diamonds with better cuts are not necessarily more expensive.

1.6.7 Making facets

Having too much information in a single plot can be overwhelming. The previous plot is already pretty busy. Facets are a nice way of spliting up plots and showing information in separate panels.

Let’s take a look at how wide these diamonds tend to be. The width in mm is given in the y column of the diamonds data frame. We’ll make a histogram first. To make a histogram, the only aesthetic we needed to specify is x.

ggplot(data = df.diamonds,
       mapping = aes(x = y)) +
  geom_histogram()
#> `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

That looks bad! Let’s pick a different value for the width of the bins in the histogram.

ggplot(data = df.diamonds, mapping = aes(x = y)) +
  geom_histogram(binwidth = 0.1)

Still bad. There seems to be an outlier diamond that happens to be almost 60 mm wide, while most of the rest is much narrower. One option would be to remove the outlier from the data before plotting it. But generally, we don’t want to make new data frames. Instead, let’s just limit what data we want to show in the plot.

ggplot(data = df.diamonds, mapping = aes(x = y)) +
  geom_histogram(binwidth = 0.1) +
  coord_cartesian(xlim = c(3, 10))

I’ve used the coord_cartesian() function to restrict the range of data to show by passing a minimum and maximum to the xlim argument. This looks better now.

Instead of histograms, we can also plot a density fitted to the distribution.

ggplot(data = df.diamonds, mapping = aes(x = y)) +
  geom_density() +
  coord_cartesian(xlim = c(3, 10))

Looks pretty similar to our histogram above! Just like we can play around with the binwidth of the histogram, we can change the smoothing bandwidth of the kernel that is used to create the histogram. Here is a histogram with a much wider bandwidth:

ggplot(data = df.diamonds, mapping = aes(x = y)) +
  geom_density(bw = 0.5) +
  coord_cartesian(xlim = c(3, 10))

We’ll learn more about how these densities are determined later in class.

I promised that this section was about making facets, right? We’re getting there! Let’s first take a look at how wide diamonds of different color are. We can use grouping to make this happen.

ggplot(data = df.diamonds,
       mapping = aes(x = y,
                     group = color,
                     fill = color)) +
  geom_density(bw = 0.2, alpha = 0.2) +
  coord_cartesian(xlim = c(3, 10))

OK! That’s a little tricky to tell apart. Notice that I’ve specified the alpha argument in the geom_density() function so that the densities in the front don’t completely hide the densities in the back. But this plot still looks too busy. Instead of grouping, let’s put the densities for the different colors, in separate panels. That’s what facetting allows you to do.

ggplot(data = df.diamonds,
       mapping = aes(x = y, fill = color)) +
  geom_density(bw = 0.2) +
  facet_grid(cols = vars(color)) +
  coord_cartesian(xlim = c(3, 10))

Now we have the densities next to each other in separate panels. I’ve removed the alpha argument since the densities aren’t overlapping anymore. To make the different panels, I used the facet_grid() function and specified that I want separate columns for the different colors (cols = vars(color)). What’s the deal with vars()? Why couldn’t we just write facet_grid(cols = color) instead? The short answer is: that’s what the function wants. The long answer is: long. (We’ll learn more about this later in the course.)

To show the facets in different rows instead of columns we simply replace cols = vars(color) with rows = vars(color).

ggplot(data = df.diamonds, mapping = aes(x = y, fill = color)) +
  geom_density(bw = 0.2) +
  facet_grid(rows = vars(color)) +
  coord_cartesian(xlim = c(3, 10))

Several aspects about this plot should be improved:

• the y-axis text is overlapping
• having both a legend and separate facet labels is redundant
• having separate fills is not really necessary here

So, what does this plot actually show us? Well, J-colored diamonds tend to be wider than D-colored diamonds. Fascinating!

Of course, we could go completely overboard with facets and groups. So let’s do it! Let’s look at how the average price (somewhat more interesting) varies as a function of color, cut, and clarity. We’ll put color on the x-axis, and make separate rows for cut and columns for clarity.

ggplot(data = df.diamonds,
       mapping = aes(y = price,
                     x = color,
                     fill = color)) +
  stat_summary(fun.y = "mean",
               geom = "bar",
               color = "black") +
  stat_summary(fun.data = "mean_cl_boot",
               geom = "linerange") +
  facet_grid(rows = vars(cut),
             cols = vars(clarity))
#> Warning: Removed 5 rows containing missing values (geom_linerange).

Figure 1.23 is stretching it in terms of how much information it presents. But it gives you a sense for how to combine the differnet bits and pieces we’ve learned so far.

1.6.8 Global, local, and setting aes()

ggplot2 allows you to specify the plot aesthetics in different ways.

ggplot(data = df.diamonds,
       mapping = aes(x = carat,
                     y = price,
                     color = color)) +
  geom_point() +
  geom_smooth(method = "lm", se = F)

Here, I’ve drawn a scatter plot of the relationship between carat and price, and I have added the best-fitting regression lines via the geom_smooth(method = "lm") call. (We will learn more about what these regression lines mean later in class.)

Because I have defined all the aesthetics at the top level (i.e. directly within the ggplot() function), the aesthetics apply to all the functions afterwards. Aesthetics defined in the ggplot() call are global. In this case, the geom_point() and the geom_smooth() functions. The geom_smooth() function produces separate best-fit regression lines for each different color.

But what if we only wanted to show one regression line instead that applies to all the data? Here is one way of doing so:

ggplot(data = df.diamonds, mapping = aes(x = carat, y = price)) +
  geom_point(mapping = aes(color = color)) +
  geom_smooth(method = "lm")

Here, I’ve moved the color aesthetic into the geom_point() function call. Now, the x and y aesthetics still apply to both the geom_point() and the geom_smooth() function call (they are global), but the color aesthetic applies only to geom_point() (it is local). Alternatively, we can simply overwrite global aesthetics within local function calls.

ggplot(data = df.diamonds, aes(x = carat, y = price, color = color)) +
  geom_point() +
  geom_smooth(method = "lm", color = "black")

Here, I’ve set color = 'black' within the geom_smooth() function, and now only one overall regression line is displayed since the global color aesthetic was overwritten in the local function call.

1.7.1 Cheatsheets

• RStudio IDE –> information about RStudio
• RMarkdown –> information about writing in RMarkdown
• RMarkdown reference –> RMarkdown reference sheet
• Data visualization –> general principles of effective graphic design
• ggplot2 –> specific information about ggplot

1.7.2 Data camp courses

• Introduction to R
• RStudio IDE 1
• RStudio IDE 2
• Communicating with data
• ggplot2 course 1
• ggplot2 course 2

1.7.3 Books and chapters

• R graphics cookbook –> quick intro to the the most common graphs
• R for Data Science book
  • Data visualization
  • Graphics for communication
• Data Visualization – A practical introduction (by Kieran Healy)
  • Look at data
  • Make a plot
  • Show the right numbers
• Fundamentals of Data Visualization –> very nice resource that goes beyond basic functionality of ggplot and focuses on how to make good figures (e.g. how to choose colors, axes, …)

1.7.4 Misc

• ggplot2 extensions –> gallery of ggplot2 extension packages
• ggplot2 gui –> ggplot2 extension package
• ggplot2 visualizations with code –> gallery of plots with code
• R Markdown in RStudio introduction
• R Markdown for class reports
• knitr in a nutshell
• styler –> RStudio add-in that re-formats code